Any ideas on what needs to be converted between what the the inpainting pipeline is expecting and what the ImageEditor is providing gratefully received. I've resolved the obvious dictionary naming conflicts for the parameters passed in, but the actual image formats seem to be the problem.

@one-lithe-rune you can refer to my changes to img2img UI for how to handle most of this.
latest version is here: #2025

Thank you for the help with this. I was very sad to have broken the buttons.

Note: We'll be moving a lot of the guts of SHARK studio elsewhere, and I don't know how much of this UI will be kept. Any ideas on consolidating features / UI representations would be massively appreciated. Let's try and simplify as we go (despite my adding a bunch of stuff for multicontrolnet...)

for example :: we dont need a tab for each feature of SD.
We can have txt2img and img2img if really needed, but inpaint/outpaint definitely dont need it.

}
I mean to rework the pipelines to reduce code overlap, and a lot of that will be coming with Studio 2.0..

Anything you have in a working state, I'll happily migrate to the new version, but just FYI that we'll be reworking a lot of the UI soon.

Note: We'll be moving a lot of the guts of SHARK studio elsewhere, and I don't know how much of this UI will be kept. Any ideas on consolidating features / UI representations would be massively appreciated. Let's try and simplify as we go (despite my adding a bunch of stuff for multicontrolnet...)

@monorimet Yeah, I was expecting that, since it at least all needs moving out of the the Stable Diffusion folder structure and rebuilding under Shark Studio for Turbine.

Ditching the tabs and having mostly one UI for SD has long been something I've thought about, ever since I did the output gallery, with its left/right split. Extending that so all the visual input image/edited image/gallery stuff is on one side of the UI, and then the other side is parameters and selection stuff for whatever you're working on, with the operation you're doing (i.e txt2img) being picked picked from a dropdown. If there were tabs it would just be for selecting which of gallery/input image/mask/intermediate image you needed to see and would be restricted that section of the UI.

The other thing I think is really needed to clean up the interface, is a revamped model manager as a separate part of the UI. It would be where you specify all the things needed for AOT compile -- so model to use for txt2img/img2img, inpaint/outpaint, vae, lora's, available sizes, big/little unet, tuned/untuned, which schedulers are sensible etc. Then in the main UI you'd just pick from the configs you'd defined there. Obviously we would need to be able to auto-generate some to get you started, and generate template ones for new models. It would also ideally tell you what was downloaded/not downloaded, built not/built for a config, and let you fire off and monitor builds without always having to do them lazily.

I assume LLM could use such a thing but with different settings. But I confess, I don't use SHARK for LLM.

Of course some of that might be moot depending on how AOT Turbine is/wants to be. I gather it can do more JIT stuff than currently?

The model manager/config stuff has been getting really itchy for me, especially with multi-LoRA being an obvious extension to what I've been working on recently. I keep running into 'and how long is a .vmfb filename going to be now?' and 'Windows has a 260 character path length limit, unless you do registry edits, doesn't it?' when I think about it.

Okay, I have inpainting 'working' again. Albeit messily.

I haven't been able to find what in my code or gradio that triggering the underlying problem that I'm having to work around (the mask layer being deleted from the ImageEditor when you trigger something that reads/use the mask). I think its possibly some queuing thing, but I dunno 🤷

Setting this as ready for review, since its 'working' and it would be good to get the other stuff in as well.

Rebased and squashed, after some rework of the queuing and progress rendering along the inpaint UI generation codepath, that allows re-enabling of output gallery updates for inpaint.

Any chance of getting this reviewed and merged?

I would bring it up to Gradio 4.9.0, but I can't get that version respect css @media queries atm, so I'm going to have to file a bug with upstream and see what comes of it before I can progress things beyond 4.8.0.